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Abstract. State estimation under sampling rate constraints is important for Net- 
worked control. To obtain the lowest possible estimator distortion under such con- 
straints, the samples must be chosen adaptively based on the trajectory of the signal 
being sampled, rather than deterministically. We treat the case of perfect observations 
at the sensor in which it measures a diffusion state process perfectly. The sensor has 
to choose causally, exactly N sampling times when it transmits samples to a supervisor 
which receives the samples without delay or distortion. Based on the causal sequence 
of samples it receives, the supervisor maintains a continuous MMSE estimate. In this 
paper we provide the optimal adaptive sampling rules to be used by the sensor that 
minimize the aggregate, finite-horizon, mean-square error distortion for scalar linear es- 
timation. For the mean squared error distortion criterion, we reduce the problem of 
joint choice of sampling rules and estimator waveforms to one of choosing the sampling 
rules alone. We find the optimal sampling times to be first hitting times of time- varying, 
double-sided and symmetric envelopes by the estimation error signal. When the state 
is Brownian motion, we characterize the optimal sampling envelopes analytically - they 
are reverse parabolic. In the case of the Ornstein-Uhlenbeck process, we provide a nu- 
merical procedure for computing these envelopes. We also characterize the performance 
of Delta sampling and furnish rules to recursively choose the successive thresholds opti- 
mally. The results of these calculations are surprising. Delta sampling performs worse 
than even the periodic sampling scheme, except possibly when the sample budget is 
quite small. Moreover, the estimation distortion due to Delta sampling decreases as the 
sample budget is increased but is bounded away from zero. 
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1. Introduction 

Networked control systems involve control loops some of whose constituent links are 
completed over data networks rather than over dedicated analog wires. In such systems, 
monitoring and control tasks have to be performed under constraints on the amount 
of information communicated to the control station. These communication constraints 
limit the rate of packet transmissions from sensor nodes to the supervisor node(s). Apart 
from the limits on transmission rates, the packets could also be delivered with delays and 
sometimes, be lost. All of these communication degradations in the networked problem 
lower performance and system design must take them into account. In this paper, we 
study only the effect of constraints on communication rates. 

Communication rate constraints can be of the following three types: 1) Average rate 
limit: This is a 'soft constraint' and calls for an upper limit on the average number of 
transmissions; 2) Minimum waiting time between transmissions: Under this type, there is 
a mandatory minimum waiting time between two successive transmissions from the same 
node; 3) Finite transmission budget: This is a 'hard constraint' and allows only up to a 
prescribed number of transmissions from the same node over a given time window. In the 
simplest application of this type of constraint, we set the constraint's window to be the 
problem's entire time horizon. In a variation, one meant to reduce the jitter in sample 
times, we partition the problem's time-horizon into multiple disjoint segments and, for 
every segment, apply the finite transmission budget constraint. 

Notice that these different types of constraints can be mixed in interesting ways. In 
this work, we will adopt the simple version of the finite transmission budget applied over 
the entire time horizon of the problem, as the communication constraint, it being natural 
for finite-horizon problems. We study a problem of state estimation which is an important 
component of distributed control and monitoring systems. Specifically, a scalar linear 
signal is continuously and fully observed at a sensor which generates a limited number 
of samples. A supervisor receives the sequence of samples and, on its basis, maintains a 
causal estimate. The restriction on the number of samples clearly limits the minimum 
estimation distortion achievable at the supervisor. The design question is: How should the 
samples be chosen by the sensor to minimize the estimation distortion ? The answer to 
this question employs the idea that: Samples should be generated only when they contain 
'sufficiently' new information. Adaptive sampling schemes, or, event-triggered sampling 
schemes as they are also called, exploit this idea and generate samples at times determined 
by the trajectory of the source signal being sampled. In contrast, deterministic sampling 
chooses sample times according to an extraneous clock. The search for efficient adaptive 
schemes leads us to a problem of optimal timing, one seeking the minimum distortion of 
the estimator. 

Under adaptive sampling, the sensor chooses sampling times based on information it 
gathers from the continuous observations; the supervisor maintains its estimate based on 
information from sample times and samples. This information pattern is non-classical 
and leads to an estimator whose error statistics are more complicated than the one under 
deterministically sampled systems. We tackle this complication, simplify optimal timing 
problem, and, show that adaptive sampling policies that apply (time-varying) thresh- 
olds on the estimation error signal achieve the minimum distortion. We also study a 
suboptimal but well-known and natural adaptive policy: Delta-sampling. We charac- 
terize completely the structure of optimal sampling, and, the performances of optimal 
and Delta-sampling policies for drift-free linear systems (the Brownian motion process). 
For processes with non-zero drift (the Ornstein-Uhlenbeck process), we characterize the 
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structure of the optimal policy but have to compute its performance, and, that of Delta- 
sampling through numerical computations. We indicate the procedures for these compu- 
tations. 

1.1. Packet rate constraints in networked control systems. Sampling rate 
constraints arise in networked control systems because of constraints on the rate of data 
packets. We will deduce this by examining the properties of packet-based communication 
schemes. In most scenarios ^ \20\ 115] , the packets are of uniform size and even when of 
variable size, have at least a few bytes of header and trailer files. These segments of the 
packet carry source and destination node addresses, a time stamp at origin, some error 
control coding, some higher layer (link and transport layers in the terminology of data 
networks) data blocks and any other bits/bytes that are essential for the functioning of 
the packet exchange scheme but which nevertheless constitute what is clearly an overhead. 
The payload or actual measurement information in the packet should then be at least 
of the same size as these 'bells and whistles'. It costs only negligibly more in terms of 
network resources, of time, or of energy to send a payload of five or ten bytes instead 
of two bits or one byte when the overhead part of the packet is already five bytes. This 
means that the samples being packetized can be quantized with very fine detail, say with 
four bytes, a rate at which the quantization noise can be ignored for low dimensional 
variables. For Markov state processes, this means that all of these bytes of payload 
can be used to specify the latest value of the state. In other words, in essentially all 
packet-based communication schemes, the right unit of communication cost is the cost 
of transmitting a single packet whether or not the payload is longer by a few bytes; 
the exact number of bits used to quantize the sample is not important. There are of 
course special situations where the quantization rate as well as the sample generation 
rate matter. An example occurs in the internet congestion control mechanism called 
Transmission Control Protocol (TCP) |14] where a node estimates the congestion state 
of a link through congestion bits added to regular data packets it receives. In this case, 
the real payload of packets is irrelevant to the congestion state, and, the information on 
the congestion state is derived from the one or two bits riding piggy-back on the data 
packets. The developments in this paper does not apply to such problems where the 
effect of quantization is important. 

State estimation problems with communication rate constraints arise in a wide variety 
of networked monitoring and control setups such as sensor networks of various kinds, 
wireless industrial monitoring and control systems, rapid prototyping using a wireless 
network, and, multi-agent robotics. A recent overview of research in Networked control 
systems including a variety of specific applications is available from the special issue 
Event-triggered sampling may also be used to model the functioning of various neural 
circuits in the nervous systems of animals. After all, the neuron is a threshold-triggered 
firing device whose operation is closely related to Delta-sampling. However, it is not 
presently clear if the communication rate constraint adopted in this paper occurs in 
Biological Neural networks. 

1.2. Relationship to previous works. The problem of choosing the time instants 
to sample sensor measurements has received early attention in the literature. Kush- 
ner |18j studied the deterministic choice of measurement times in a discrete-time, finite- 
horizon LQG optimal control problem. He found that controls can be chosen like in a 
conventional LQG control problem with fixed deterministic sampling instants. Using this 
fact, he shows that the best (deterministic) sampling schedule can be found by solving 
a nonlinear optimization problem. Skafidas & Nerode |29] also adopt the same setting 
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but allow the sensor measurement times to be chosen online by the controller. Their 
conclusion is that the optimal choice of measurement times can be made offline. Their 
simplified scheduling problem is the same as Kushner's deterministic one. 

The Sensor scheduling problem is a generalization of the problem of choosing mea- 
surement times which has been studied for estimation, detection and control tasks |22l, 
I21L [5|, 130] . The problem asks for online schedules of when to gather measurements from 
different sensors available; in some setups, at most one sensor can be scheduled at a time. 
However, the information pattern for this problem is the same as the the works of Kush- 
ner and of Skafidas & Nerode. The notable fact about this information pattern is that the 
fiow of data from sensors to their recipients namely, controllers, estimators etc., is regu- 
lated by the recipient. Such sensor sampling is of the "pull" type. An alternative is the 
"push" type of sampling where sensor itself regulates the fiow of its data. When only one 
sensor is available, it has more information than the controller and hence, its conclusions 
on when to communicate its measurements can potentially improve performance. 

Astrom & Bernhardsson |4] treat a minimum variance control problem with the push 
type of sampling. The control consists of impulses which reset the state to the origin, but 
there is an upper limit on the average rate at which impulses can be applied. Under such 
a constraint, the design asks for a schedule of the application times for the impulses. For 
scalar Gaussian diffusions, they perform explicit calculations to show that the application 
of impulses using Delta sampling, namely triggering by first hitting of fixed thresholds is 
more efflcient than the periodic application of impulses. Their conclusions are relevant 
for us because, in our estimation problem, the estimation error signal is reset to zero at 
sampling times, and so, sampling in our problem corresponds in their problem to reset- 
ting the state by applying an impulse. With this correspondence, further conclusions can 
be drawn about the problem of repeated sampling for estimation over an infinite horizon. 
The thesis |25] of the first author of this paper shows that the Delta sampling scheme is 
indeed optimal for the infinite horizon problem of |4]. Recently, Henningsson et.al. |12] 
have extended the results of [3| and of |25] to include a communication constraint con- 
taining both an average rate constraint and an minimum waiting time between sampling 
instants. We should also mention the interesting fact that Delta sampling of a Wiener 
process has been found by Berger [6] to be the compression scheme achieving the Infor- 
mation theoretic Rate-Distortion limit. Theere is also the work of Lazar & Toth |19] 
which attempts to extend the Shannon-Nyquist sampling theorem to the case of Delta 
sampling of deterministic signals. 

The work of Imer & Basar |13j considers the finite-horizon adaptive sampling problem 
but in a discrete time setting. They use the optimal form of the estimator under adaptive 
sampling with a partial proof (see proposition 1 of |13j ) and describe dynamic program- 
ming equations to be satisfied by the optimal sampling policy. We should point out that 
their sampling scheme for the quadratic distortion namely the time-varying symmetric 
threshold for the estimation error, is indeed the optimal form of the sampling policy - this 
can be proved, along the lines of our work, by converting the optimal stopping problem 
for minimum estimation distortion into one with just a terminal cost or reward. 

The study of optimal adaptive sampling timing leads to Optimal stopping problems of 
Stochastic control, or equivalently as we have already seen, to impulse control problems. 
The information pattern of adaptive sampling complicates the picture but methods of 
solving multiple stopping time problems of standard form which are available in the 
literature 181 are indeed useful. 



The work reported in this paper has been announced previously in |25L I26L I27L I28j . 
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1.3. Contributions and outline of the paper. For the finite horizon state esti- 
mation problem, we cast the search for efficient sampling rules as sequential optimization 
problem over a fixed number of causal sampling times. This we do in section [2] where, we 
formulate an optimal multiple stopping problem with the aggregate quadratic distortion 
over the finite time horizon as its objective function. Although the information pattern 
for the resulting sequential decision problem is non-classical, we use the facts that the 
state evolution is linear and the quadratic error is an even function, and show that the 
optimal sampling policy produces a simple and intuitive estimator. We also suggest some 
modified design strategies for nonlinear and partially observed cases. 

In section |3j we take the simplified optimal multiple stopping problem and solve it 
explicitly when the state is the (controlled) Brownian motion process. The optimal sam- 
pling policies are first hitting times of time-varying envelopes by the estimation error 
signal. Our analytical solution shows that for each of the sampling times, the triggering 
envelopes are symmetric around zero and diminish monotonically in a reverse-parabolic 
fashion as time nears the end of the horizon. We also describe analytically, the perfor- 
mance of the class of modified Delta sampling rules in which, we apply different values 
of the threshold A for the different sampling times. We point out a simple and recursive 
procedure for choosing the most efficient of these Delta sampling policies. 

For the Ornstein-Uhlenbeck process, in section |4| we derive Dynamic programming 
equations for optimal sampling policy. We compute the solution to these equations nu- 
merically. We are not able to say whether an explicit analytic solution like for the 
Brownian motion process is possible. The optimal sampling times are first hitting times 
of time-varying envelopes by the estimation error signal. These envelopes are symmetric 
around zero and diminish monotonically as time nears the end of the horizon. Also de- 
rived are the equations governing the performance of modified Delta sampling rules and 
the most efficient among them is found through a numerical search. Finally, in section [5} 
we conclude and speculate on extensions to this work for other estimation, control and 
detection problems. 

2. MMSE estimation and optimal sampling 

Under a deterministic time-table for the sampling instants, the Minimum mean square 
error (MMSE) reconstruction for linear systems is well-known and is straightforward 
to describe - it is the Kalman filter with intermittent but perfect observations. The 
error variance of the MMSE estimate obeys the standard Riccati equation. In Delta- 
sampling |23L IIOL lllj . also called Delta-modulation, a new sample is generated when 
the source signal moves away from the previously generated sample value by a distance 
A. By this rule, between successive sample times, the source signal lies within a ball 
of radius A which is centered at the sample value at the earlier of the two times. Such 
information being available about the behaviour of the source signal between sample 
instants is typical of adaptive sampling schemes but never of deterministic sampling 
schemes. A reconstruction, causal or not, of a signal based on its samples generated 
adaptively leads to an estimator structure different from that of deterministic sampling. 
In this section, we will describe the modiffed structure for a causal estimator minimizing 
the MMSE. 

We will also set up an optimization problem where we seek an adaptive sampling 
policy minimizing the distortion of the MMSE estimator subject to a limit on the number 
of samples. By examining some simple symmetry in the statistics of linear systems 
perturbed by white noise, we will obtain simplifications for the MMSE estimator under 
optimal sampling. Consider a state process Xt which is a (possibly controlled) scalar 
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(a) Adaptive sampling for real-time estimation (b) MMSE estimate under a time- varying thresh- 
old policy 

Figure 1. Subfigure (a) depicts the setup for the MMSE estimation based 
on samples arriving at a hmited rate. Subfigure (b) depicts the difference 
between estimators for Adaptive samphng and deterministic samphng. 



hnear diffusion. It evolves according to the SDE: 

(1) dxt = axtdt + hdBt + Utdt^ xq = x, 

where, Bt is a standard Brownian motion process. The control process Ut is ROLL and 
of course measurable with respect to the x-process. In fact, the feedback form of Ut is 
restricted to depend on the sampled information only; we will describe this subsequently. 
We assume that the drift coefficient a, the noise coefficient 6 7^ 0, and, the initial value x 
are known. Now, we will dwell upon sampling and the estimation process. 

The state is sampled at instants {Ti}j>Q which are stopping times w.r.t. the a;-process. 
The samples along with the sampling times are encapsulated into data packets and sent 
over the communication link. This link, while limited in its average throughput, does not 
require any minimum interval between packets; It also delivers the packets it handles, 
reliably and with negligible delay. Essential to this abstraction is an assumption that the 
transactions on the data network happen on a time scale faster that the dynamics of the 
plant. Formally, we require that the length of data packets be much smaller than the 
time-constant of the plant: ^. The samples are of scalars or low-dimensional vectors and 
so quantization can be ignored. 

Let Tit represent the data contained in the packets received at the estimator. It is a 
growing sequence containing the sequence of samples received and also their timing: 



(Tj, Xj 



Notice that the binary-valued process l{T,<t} 



r, < t 



is measurable w.r.t. 



t ■ 



The MMSE 



estimate Xt is based on knowledge of the multiple sampling policy and all the information 
contained in the output of the sampler and so it can be written as: 



Xt = E 



Xt 



^t 



The control signal Ut is measurable w.r.t. J-"^ 



certainty-equivalence type as depicted in figure 
measurable w.r.t. J-'^. The exact form of the feedback control is not important for our 



Typically it is restricted to be of the 
1(a) In that case Ut is, in addition. 
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work, but the requirement that the estimator knows the feedback control policy (and 
so has access to the control waveform Mq) is essential. With this assumption, at the 
estimator, the control waveform is a known additive component to the state evolution 
and can be subtracted out. Hence, there is no loss of generality in considering only the 
uncontrolled plant. 



2.1. MMSE estimation under deterministic sampling. Consider now a deter- 
ministic scheme for choosing the sampling times. Let the sequence of non-negative and 
increasing sampling times be: 

D = {c?o = 0,rfi,...}, 

where, the times di are all statistically independent of all data about the state received 
after time zero. They could at most be correlated to the initial value of the state xq. 

We will now describe the MMSE estimate and its variance. Consider a time t in the 
semi-open interval [di,dij^i). We have: 



= E 
= E 
= E 
= E 



di<t< di+i, di,Xd, 

dij Xdj 



0>j<i 



where, we have used the Markov property of the state process and the mutual indepen- 
dence, conditional on xq, of the state and the sequence V. Furthermore, 



Xt = E 



e'^^'-'^^'^Xd- 



di 







[ e'^^'-'^Usds 


dii Xdi 


'di 





Thus, under deterministic sampling, the MMSE estimate obeys a linear ODE with jumps 
at the sampling times. 



dxt 
~dt 



axt + Ut, for t ^V, and, Xt = Xt, if t eV. 



The variance pt = E, ^{xt — x*)^] is given by the well-known Riccati equation: 
dpt 



dt 



2apt + b, for t ^ V, and, pt = 0, if t G V. 



The above description for the MMSE estimate and its variance is valid even when the 
sampling times are random provided that they are independent of the state process. Then, 
the evolution equations for the MMSE estimate and its error statistics remain independent 
of the policy for choosing the sampling times; the solution to these equations merely get 
reset with jumps at these times. On the other hand, adaptive sampling modifies the 
evolution of the the MMSE estimator as we will see next. 
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2.2. The MMSE estimate under adaptive sampling. Under adaptive sampling 
policies, the sampling times are stopping times w.r.t. the state process. Between sample 
times, an estimate of the state using any distortion criterion is an estimate up to a 
stopping time and this is the crucial difference from deterministic sampling. At time t 
within the sampling interval [rj, Tj+i), the MMSE estimate is given by: 



Xt = E 
= E 
= E 



Si 

Ti<t< Ti+i, |(r,-,x^J 

Ti < t < Tj+i, Tj, Xn 



0>3<i 



Xr- + E 



Xi X-J-. 



t T,i <Z Tj, Tj, Xj 



Similarly, its variance pt can be written as 

\2 



Pt = E 



{Xt - XtY 



Between samples, the MMSE estimate is an estimate up to a stopping time because the 
difference of two stopping times is also a stopping time. In general, it is different from 
the MMSE estimate under deterministic sampling (see appendix |5]) . This simply means 
that in addition to the information contained in previous samples and sample times, 
there are extra partial observations about the state. This information is the fact that 
the next stopping time r^+i has not arrived. Thus, in adaptive schemes, the evolution 
of the MMSE estimator is dependent on the sampling policy. This potentially opens the 
possibility of a timing channel [2\ for the MMSE estimator. 

Figure 1(b) describes a particular (sub-optimal) scheme for picking a single sample. 
There are two time- varying thresholds for the state signal, an upper one and a lower one. 
The initial state is zero and within the two thresholds. The earliest time within [0,T], 
when the state exits the zone between the thresholds is the sample time. The evolution 
of MMSE estimator is dictated by the shape of the thresholds thus utilizing information 
available via the timing channel. 



2.3. An optimal stopping problem. We formalize a problem of sampling for op- 
timal estimation over a finite horizon. On the interval [0, T], we seek for the state process 
([T]), with the initial condition xq, an increasing and causal sequence of at most sampling 
times {ti, . . . , tn}, to minimize the aggregate squared error distortion: 



(2) J (T, A^) = E 



Notice that the distortion measure does not depend on the initial value of the state 
because, it operates on the error signal (e^ = Xt — Xt) which is zero at time zero. Notice 
also that the communication constraint is captured by an upper limit on the number of 
samples. In this formulation, we do not get any reward for using fewer samples than the 
budgeted limit. 

The optimal sampling times can be chosen one at a time using a nested sequence of 
solutions to optimal single stopping time problems. This is because, for a sampling time 
Tj+i which succeeds the time Tj, using a knowledge of how to choose Tj+i optimally, we 
can obtain an optimal choice for Ti by solving over [0, T] the optimal single stopping time 
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problem 



ess inf E 

r>0 



[Xs 



'^ds + J*{T- Ti, N-i) 



where, J* (T — Ti,N — i) is the minimum distortion obtained by choosing N — i sample 
times {Tj_|_i, . . . ,tn} over the interval [rj,T]. The best choice for the terminal sampling 
time tn is based on a solving a single stopping problem. Hence we can inductively find 
the best policies for all earlier sampling times. Without loss of generality, we can examine 
the optimal choice of the first sampling time ti and drop the subscript 1 in the rest of 
this section. 

2.3.1. An optimal stopping problem with non-standard costs. The sampling problem 
is to choose a single JF^^'-stopping time r on [0, T] to minimize 



F(T,i; 



E 



{xs - Xsfds + J*{T-T,N-1] 



where, 



J* 



ess inf E [ J (T 

{t2,...,tn} 



T,N-1)]. 



The cost F has a terminal component which is standard. The terminal part is a function 
only of the number of remaining samples N — 1, and, the amount of time left T — t 
because, given r, the terminal part is independent of x^. 

But, the cost has a running integral up to the stopping time whose integrand depends 
on the stopping policy. If, for every stopping policy, we can somehow determine the wave- 
form {xt}J^Q, then, we can solve the optimization problem by solving a standard optimal 
stopping problem. To solve such typical stopping problems, we can use the process of 
minimum expected distortion, the so-called Snell envelope (St) (see |17] Appendix D and 
also 1241): 



St 



ess inf E 

T>t 



[Xc Xq 



[Xs 



Xc 



'^ds + .r{T-T,N-l) 



'ds + ess inf 

T>t 



E 



{xs - Xsfds + J*{T-T,N-1) 



Ut 



Xt 



Then, the smallest time when the cost of stopping does not exceed the Snell envelope is 
an optimal stopping time. Since the Snell envelope depends only on the current value 
of the state and the current time, we get a simple threshold solution for our problem. 
Now, we will use some properties of the state process and of the squared error distortion 
criterion to bring our stopping problem into standard form. 

2.4. Linearity simplifies the MMSE estimate under optimal adaptive sam- 
pling. The sampling problem for state estimation has been cast so far as a non-standard 
optimal stopping problem with the filter Xt being dependent on the stopping rule being 
optimized. Now, we will look at this optimization as one over the stopping rule as well as 
a deterministic waveform {^t}^o which the estimator uses to represent Xt- For example, 
the supervisor may want to use a piece-wise linear waveform to keep track of Xt until the 
sampling time but use the least squares estimate Xt after the sample has been received. 
With the arbitrarily chosen piecewise deterministic estimate waveform, the distortion 
becomes: 



E 



[Xs 



isfds + J* {T-T,N-1) 
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All the sensor needs to do now is to tailor its stopping policy T to minimize the aggregate 
distortion between the state and the supervisor's estimate process. For a fixed estimator 
C,t, let be an optimal stopping rule which produces the stopping time r* and 

minimizes J^, i.e. 

^ ({6}f=o) = ^ ({6}f=o) > {6}f=o) < ^ {6}f=o) > V T. 

Lemma 1. Let {Cltlo ^ waveform that minimizes ^. Then, it follows that the pair 
^*({0^o)' {Ot=o) minimizes . The optimal waveform {^t}J^Q has the property 
that it is also the MMSE estimate corresponding to the stopping policy T*({^^}^q).- 

Ct =■ E[xt\T*>t,]. 

Proof: If the lemma were not true, we could achieve a lower cost by retaining the 
sampling policy T* ({^^}^q) and using the conditional mean it generates as the waveform 
at the estimator. ■ 

This lemma says that if a pair ^^*({C}t=o)' {C}t=o) minimizes J^, then, {^f }^q is 
almost surely equal to the MMSE estimate of Xt up to the stopping time generated by 
Hence, 



Now, we will use the linearity of the state evolution to narrow down the kind of pairs 
of stopping rule and waveform that minimize Jif. Conditional on the initial condition: 
the unobserved Xt has at all times, an even density function centered at 

xe"' + / e^^'-'^Us ds. 



This follows from the linearity of the Fokker-Planck operator [26]. Now, it turns out that 
combining the waveform 

T 



Jo ) t=0 



with the best stopping rule for the waveform {Q}^^^ does not increase the cost ! This is 
because the "innovations" process 

Xt- [ xe"^* + ! e'^'-'-'^sds 



has the same statistics as its negative 



-xt+ (^xe"' + e^^'-'^sd^ . 
Since the conditional mean is the minimizing waveform a.s., we have that 

Ct = 2x6"' + 2 [ e^^'-'^sds-Cf 
Jo 

Because of this, we can be assured of the validity of the following useful Lemma which is 
the fruit of our labours in this section. 
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Lemma 2. The optimal MMSE estimate is the mean of the Fokker- Planck equation 
and is given by: 



E[xt\T*>t,] =■ Ct = a;e"*+ [ e'^^^-^^w.rfs, VtG[0,T] 

Jo 



Thus we have shown that under the optimal adaptive samphng strategy, the statistics 
of the state given the stream of time-stamped samples is such that the conditional mean 
is the same as that under deterministic sampling. However, because of the threshold 
nature of the optimal stopping policy, the higher moments of the estimation error are 
strictly lesser than those under any deterministic sampling scheme. Nevertheless we have 
succeeded in reducing the problem of optimal timing of samples to an optimal multiple 
stopping problem with an objective function of standard form. 

2.5. Extensions to nonlinear and partially observed systems. For nonlinear 
plants, it is possible that under optimal sampling, the MMSE estimator is no longer the 
mean of the Fokker-Planck equation: 

= E Xt Tlatest, ^n^^^^t > wllCrC, Tiatest = SUp {t^ < t} . 

To obtain a tractable optimization problem we can restrict the kind of estimator wave- 
forms allowed at the control station. Using the Fokker-Planck mean above leads to a 
tractable stopping problem as does using the zero-order hold waveform: 

However, even a slightly more general piece-wise constant estimate: 

= h (a^Tiatesf '^latest) ■ 

leads to a stopping problem of non-standard form. 

When the plant sensor has a noisy, and in the vector case case, partial observations, 
the sampling problem remains unsolved. The important question now is: What signal 
at the sensor should be sampled ? Should the raw sensor measurements be sampled and 
transmitted, or, is it profitable to process them first ? We propose a solution with a 
separation into local filtering and sampling. Accordingly, the sensor should compute a 
continuous filter for the state. The sufficient statistics for this filter should take the role 
of the state variable. This means that the sensor should transmit current samples of 
the sufficient statistics at, sampling times that are stopping times w.r.t. the sufficient 
statistics process. 

In the case of a scalar linear system with observations corrupted by white noise, the 
local Kalman filter at the sensor f^^"*"'' plays the role of the state signal. The Kalman 
filter obeys a linear evolution equation and so the optimal sampling policies presented 
in this paper should be valid. In the rest of the paper, we will investigate and solve 
the sampling problem first for the Brownian motion process and then for the Ornstein- 
Uhlenbeck process. 



3. Sampling Brownian motion 

The sampling problem for Brownian motion with a control term added to the drift is 
no different from the problem without it. This is because the control process {wtjoo 
measurable w.r.t. J-"^, whether it is a deterministic feed-forward term or feedback based 
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on the sampled information. Thus, for the estimation problem, we can safely set the 
control term to be identically zero: 



dxt = bdBt, Xq = X. 



The diffusion coefficient b can be assumed to be unity. If it is not, we can simply scale 
the time variable and in ^-time, the process obeys a SDE driven by a Brownian motion 
with a unit diffusion coefficient. We study the sampling problem under the assumption 
that the initial state is known to the MMSE estimator. Note that as per Lemma|2] the 
MMSE estimate for this process is a zero order hold type reconstruction from the received 
sequence of samples. 

In this chapter, we study three important classes of sampling. The optimal determin- 
istic one is traditionally used and it provides an upper bound on the minimum distortion 
possible. The ffist adaptive scheme we study is Delta sampling which is based on ffist 
hitting times of symmetric levels by the error process. Finally, we completely charac- 
terize the optimal stopping scheme by recursively solving an optimal multiple stopping 
problem. 



3.1. Optimal deterministic sampling. Given that the initial value of the error 
signal is zero, we will show through induction that uniform sampling on the interval [0, T] 
is the optimal deterministic choice of samples. Set the deterministic set of sample times 
to be: 



V = {di, d2, . . . ,dj^ \ < di < T, di > for i = 2, . . . , A^} . 



Then, the distortion takes the form: 

rdi pd2 pT 

Joeter (T, N) = E (x^ — Xs) ds + E (x^ — Xg) ds + . . . + E — Xs) ds. 

Jo J di J dff 

Consider the situation of having to choose exactly one sample over the interval [Ti,T2] 
with the supervisor knowing the state at time Ti. The best choice of the sample time 
which minimizes the cost Jneter (T2 — Ti, 1) is the midpoint |(T2 + Ti) of the given 
interval. We propose that the optimal choice of iV — 1 deterministic samples over [Ti, T2] 
is the uniform one: 
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which leads to a distortion equaUing 37^(^2 — ^i)^- Now, we carry out the induction step 
to obtain the minimum distortion over the set of samphng times over [Ti,T2] to be: 

min JDeter{T2-Ti, N) = mini / {xs - Xgfds + min Joeter {T2 - Ti - di, N~\ 

{di,d2,---dN} di \^Jq {d2,d2,---dN} 



. di^ {T2 -Ti-di 
mm < h 



rfi I 2 2N 

Ndl + dl- 2d2 {T2 - Ti) + (T2 - Tr] 



mm 



di 1 2N 

{N + 1) {d, - ^ (T^ - to) ' + (1 - (T2 - T,y 



mm 

di 



2N 



2(A^ + 1 



-(T2-T1 



>2 



the minimum being achieved for di = (T2 — Ti). This proves the assertion about 
the optimahty of uniform samphng among all deterministic schemes provided that the 
supervisor knows the value of the state at the start time. 

3.2. Optimal Delta sampling. As described before, Delta sampling is a simple 
event-triggered sampling scheme which generates a new sample whenever the input signal 
differs from the estimate by a pre-specified threshold. Delta sampling is really meant for 
infinite horizon problems and it produces inter-sample intervals that are unbounded. 
Since we have on our hands a finite horizon problem, we will use a time-out at the end 
time of the problem's horizon. To make the most of this class of rules, we allow the 
thresholds to vary with the past history of sample times. This way, the sample times are 
chosen based on information available only ay the sensor while, the sequence of thresholds 
which characterize sampling times depends on information available to the supervisor as 
well as the sensor. 

More precisely, at any sampling time as well as at the start of the horizon, the thresh- 
old for the next sampling time is chosen. This choice is allowed to depend on the number 
of samples remaining as well as the amount of time left till the end of the horizon. We set 
To = 0, and define thresholds and sampling times times recursively. The threshold for the 
i^^ sampling time is allowed to depend on the values of the previous sampling times, and 
so it is measurable w.r.t. jF^p. Assume that we are given the policy for choosing causally 
a sequence of non-negative thresholds ... , ^at}. Then, for i = 1, 2, . . . , A^, we can 

characterize the sampling times {Ci, C25 • • • , Ca^} as follows: 



Ti^s, =mi{t:t> Ti_i^s,^^, \xt - > 6^} 

Ci = min {Ti^s,, T} = n^s, A 



The first threshold 5i depends only on the length of the time remaining at the start of 
the horizon namely, T. 

The optimal thresholds can be chosen one at a time using solutions to a nested 
sequence of optimization problems each with a single threshold as its decision variable. 
This is because, for a sampling time Qj^i succeeding the time Ci, using a knowledge of 
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A' - A 





(a) All three samples used 



(b) Only one sample used 



Figure 2. Estimation error signal and the optimal fixed thresholds for 
the Brownian motion process when up to three samples are permitted. 
Formally speaking, any budgeted samples not generated before the end 
time are made at the end time, an event which happens with non-zero 
probability. 



how to choose Ci+i optimally, we can obtain an optimal choice for Q by solving over [0, T] 
the optimization problem: 



inf E 

<5«>o 



where the cost function J^hresh — Q, N — i) is the minimum aggregated distortion 
over [T — (i,T] achievable using at most N — i samples generated using thresholds for 
the magnitude of the error signal. Hence, if we know how to generate the last sample 
efficiently, we can inductively figure out rules governing efficient thresholds for earlier 
sampling times. 

3.2.1. Optimal level for a single sample. We drop the subscript for the terminal 
sample time: 



Ts = inf {t : \xt - Xt\ = 6} 



and its corresponding threshold 6. Here, 5 is a threshold independent of the data acquired 
after time 0. Our goal is to compute the performance measure for any non- negative choice 
of the threshold and then select the one that minimizes the estimation distortion: 



JThresh (T, 1) (5) = E 



xlds + {xs- XrsArfds 

JtsAT 
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By using iterated expectations on the second term, we get: 



J' 



Thresh 



(T, 1) (5) 



(3) 



E 
E 
E 
E 



TsAT 



xlds + E 


/ < 


IJtsAT 


xlds + / 


^ E[(. 






xlds + / 







{Xs — Xrg/\T) ds Ts A T, XrgAT 

XtsatY\ Ts a T,Xr,AT] ds 
[s — A T) ds 



rrsfsT 1 

xlds + -[{T - Tst]- 



We have thus reduced the distortion measure to a standard form with a running cost and 
a terminal cost. We will now take some further steps and reduce it one with a terminal 
part alone. Notice that: 

d [{T - t) xl] = -xfdt + 2 (T - t) xtdxt + {T - t) dt, 

which leads to the following representation for the running cost term: 



E 



(4) 



- rrsAT 




/ x^ds 


= E 


Jo 





















iT-rsAT)x' + 



T2 



-(T-TsATY 



E 
E 



X. 



rsAT 
2 



iT-rsr + l[iT-rsr\ 



5\T - n)- 



1 



Note that equation |4] is valid even if we replace with a random time that is a stopping 
time w.r.t. the x-process. Thus, the cost ([s]) becomes: 

(5) JThresh (T, 1) (5) = Y - 5'E [{T - rsY] . 

If we can describe the dependence of the expected residual time E[(T — r^)^] on the 
threshold 6, then, we can parametrize the cost purely in terms of 6. Had we known the 
PDF of Ts the computation of the expectation of the difference {T — ts)~^ would have been 
easy. Unfortunately the PDF of the hitting time ts does not have a closed form solution. 
There exists a series representation we can find in page 99 of Karatzas and Shreve 
which is: 



Ut) = s 




7rt3 



This series is not integrable and so it cannot meet our needs. Instead we compute the 
moment generating function of (T — ts)^ and thereby compute the expected distortion. 
These computations are carried out in appendix B. In particular, equation 17 gives the 
expression: 



Thresh 



(T, 1) (A) 



1 + 



32A2 



71 

4A 



vr 
A2 



E 

fc>0 



-11 



g-(2fc+l)2A 
(2fc+ 1)3 



where, A = Our parametrization in terms of the parameter A reveals some structural 
information about the solution. Firstly, note that the length of the time horizon does not 
really matter. The form of the graph of JThresh {T, 1) as a function of the variable A is 
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invariant w.r.t. T. It is merely scaled by the factor The behaviour of the distortion 
as A is varied can be seen in figure |3| The minimal distortion incurred turns out to be: 



T2 



0.3952- 



cr 



this being achieved by the choice: 5* = 0.9391vT. As compared to deterministic sam- 
pling, whose optimum performance is 0.5^, we realize that we have slightly more than 
20% improvement by using the optimum thresholding scheme. 

How often does the Delta sampler actually generate a sample ? To determine that, we 
need to compute the probability that the estimation error signal breaches the threshold 
before the end time T. Equation 19 provides the answer: 98%. Note that this average 



sampling rate of the optimal Delta sampler is independent of the length of the time 
horizon. 

We have provided a complete characterization of the performance of the Delta sampler 
with one allowed sample. In what follows, we will characterize the behaviour of multiple 
Delta sampling. In doing so, we will hit upon the remarkable fact that if the sensor is 
allowed two or more samples, it is actually more efficient to sample deterministically. 

3.2.2. Multiple delta sampling. Like in the single sample case, we will show that the 
expected distortion over [0, T] given at most samples is of the form 

Let Ts be the level-crossing time as before. Then, given a positive real number a, consider 
the following cost: 



T(T,a,5)=E 



/ xlds + a[{T - Ts)+\ 
Jo 



Using the same technique as in the single sample case (precisely, the calculations between 
and including equations |3| |5]), we get: 

T2 /I \ r 



T (T, a, 6) = ^- 6'E [(T - r^)^] - ( - - a 1 E 



[{T 



The evaluation 



This demands an evaluation of the second moment: E 

can be done similarly to the one for the first moment and is also described in appendix 
B. The results of these calculations let us write (equation [T8|: 



T{T,a,S) 



Y 



m + 



— a 



^(A) 



where, A 



and we define the functions 



0(A)^1 + 



32A2 



and. 



^(A)= 



5n^ 
96A2 



71 

4A 



TT 

2A 



71 

A2 



ip as follows: 



E 

fc>0 



(2A; + 1)3 



2 + 



16 



k>0 



(_l)fce-(2fc+i)= 



(2A; + 1) 



The choice of A that minimizes the cost T can be determined by performing a grid search 
for the minimum of the scalar function 0(A) + — ^(A). Since this sum is a fixed 
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Plot of 2 J(Tj_)AT^ 

0.9 r 




0.1 ' ' ' ' 

5 10 15 

A = T7E^/86^ 



Figure 3. We show the estimation distortion dues to Delta samphng as 
a function of the threshold used. Notice that for a fixed A, the distortion 
decreases steadily as the maximum number of samples remaining(N) grows. 
The distortion however never reaches zero. The minimum distortion reaches 
its lower limit of 0.287^. 



function, we conclude that the minimum cost is a fixed percentage of ^ exactly as in the 
case of the single sample. This property of this optimization problem is what enables us 
to determine optimal multiple Delta samplers by induction. 

Let us return to the distortion due to N samples generated using a Delta sampler, 
with the budget N being at least 2. If we have determined the optimal Delta samplers 
for utilizing a budget of — 1 or less, then the optimal distortion with a budget of 
samples takes the form: 



J; 



Thresh 



(T, N) = inf E 



CiV 



{Xs - Xsfds + JThresh - CiV, N - 1) 



inf E 



Ts AT 



[X. 



Xa 



Suppose the sensor is allowed to generate absolutely no samples at all. Then the distortion 
at the supervisor will be: We know that the minimum distortion due to using a single 

sample is a fixed fraction of ^ namely: Ci^. We will now deduce by mathematical 
induction that the minimum distortions possible with higher sample budgets are also in 
the form of fractions of Let the positive coefficient Ck stand for the the hypothetical 
fraction whose product with ^ is the minimum distortion JThresh Continuing the 

previous set of equations, we get: 



JThresh (T, N) 



inf E 



[Xs 



X a 



inf^ T (T, cjv_i,5^) , 

inf <! 0(Ajv) + 



rp2 

~2 



''ds + cn-i[{T - Ts^)^]' 



CN-1 



Because of the scale- free nature of the functions 4>,ip,'we have proved that the minimum 
distortion is indeed a fixed fraction of Figure 3 depicts the behaviour of the sum 
functions to be considered for efficient multiple Delta sampling. The last equation gives 
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Prabability of sample generation as per Delta sampling Sample budget utilization by optimal Delta sampling 

, , , , , 1 6 I , , , , r 




2 4 6 8 10 5 10 15 20 25 

■/^ _ Number of allowed samples: N 

(a) Firing rate of Delta sampling (b) Average rate of sample generation 



Figure 4. Subfigure (a) shows the the the probabihty that a sample is 
generated (S) as a function of the parameter A, which is is inversely re- 
lated to the square of the threshold 5. Subfigure (b) demonstrates the fact 
that delta sampling is meant for repeated sampling over infinite horizons. 
The utilization of the sample budget by the optimal delta sampling scheme 
utilizes is not monotonic with the size of the budget and is actually some- 
what counter- intuitive. Over any fixed finite horizon, it uses fewer than six 
samples on average. 

us the following recursion: 



(6) 





= inf {0(A) + (0.5 -c,_i)^(A)}, 

A 


Pk 


= — , and, 

2^2X1' 


K 


= arginf {0(A) + (0.5 - Ck-i)^{\)] , 

A 


51 


= PN-k+l^/T — (k-1- 



Now we determine the expected sampling rate of the optimal Delta sampler. Let be 
the random number of samples generated before T by the Delta sampler with a budget 
of k samples. Then almost surely, equals the number of threshold crossings generated 
by this sampler. Clearly, we have the bounds: < < fc. Also, under optimal Delta 
sampling, the statistics of the sampling rate do not depend on the length of the time- 
interval T as long as the latter is nonzero. This gives us the recursion: 

(7) E [Hfc] = O.F[rs^>T] + (1 + E[E,^^]) ■ P[r,^ < T] . 

We have derived the performance characteristics of optimal Delta sampling with a fixed 
sample budget. The parameters describing the performance are tabulated in [T] for small 
values of the budget. To understand the behaviour of optimal Delta sampling when the 



N 


1 


2 


3 


4 


5 


cn 


0.3953 


0.3471 


0.3219 


0.3078 


0.2995 


PN 


0.9391 


0.8743 


0.8401 


0.8208 


0.8094 




0.9767 


1.9306 


2.8622 


3.7541 


4.4803 



Table 1. Characteristics of Optimal Multiple Delta sampling for small 
values of the sample budget. 

the sample budget is larger than five, look at subfigure (b) in |4] as well as figure [6] The 
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minimum distortion decreases with increasing sample budgets but it does not approach 
zero. It stagnates at approximately 0.3^ no matter how large a budget is provided. The 
expected number of samples is not monotonic in its dependence on the budget. It settles 
to a value close to 4.5. Clearly, Delta sampling which is optimal for the infinite horizon 
version of the estimation problem is far from optimal in the finite horizon version. In fact 
if the sample budget is at least two, then, even deterministic sampling performs better. 
There is hence a need for determining the optimal sampling policy. 

In optimal Delta sampling, the sensor chooses a sequence of thresholds to be applied 
on the estimation error signal. The choice of a particular threshold is made at the time 
of the previous sample and is allowed to depend on the past history of sample times. 
Suppose now that the sensor is allowed to modify this choice causally and continuously 
at all time instants. Then we get a more general class of sampling policies with a family 
of continuously varying envelopes for the estimation error signal. This class of policies 
contains the optimal sampling policy which achieves the minimum possible distortion. 
Next, we will obtain the optimal family of envelopes by studying the problem of minimum 
distortion as an optimal multiple stopping problem. 

3.3. Optimal sampling. Now, we go after the optimal multiple sampling policy. 
Consider the non-decreasing sequence: {ri, r2, . . . , r^^} with each element lying within 
[0, T]. For this to be a valid sequence of sampling times, its elements have to be stopping 
times w.r.t. the x-process. We will look for the best choice of these times through the 
optimization: 



J*(T, N) 



ess inf E 

{Ti,r2,...,T]v} 



Tl />T2 

2, 



Xs ds + 



ds-\ h 



[Xq Xq- 



"ds 



Xr ) ds 



The solution to this optimization parallels the developments for Delta sampling. In partic- 
ular, the minimum distortion obtained by optimal sampling will turn out to be a fraction 



of We will recursively obtain optimal sampling policies by utilizing the solution to 
the following optimal (single) stopping problem concerning the objective function %: 



ess inf x (T, /3, r) = ess inf E 



where, r is a stopping time w.r.t. the x-process that lies in the interval [0,T], and, /? is 
a positive real number. We reduce this stopping problem into one having just a terminal 
cost using the calculations between and including equations |3} [5] 

X (T, /?, r) = ^ - E {2x\ (T - r) + (1 - /3) [(T - r)]^] , 

which can be minimized by solving the following optimal stopping problem: 

ess sup E \lxl (T - r) + (1 - /3) [(T - r)]^] 

r 

This stopping problem can be solved explicitly by determining it Snell envelope process. 
We look for a function g (x, t) which satisfies the Free boundary PDE system: 



\9xx + 5't = 0, and 



g{x,t)>1x' (T-t) + (l-/3) {T-ty 
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Given a solution g, consider the process: 

St=g {xt,t) . 

This is in fact the Snell envelope. To see that, fix a deterministic time t within [0, T] and 
verify using Ito's formula that: 



E[Sr{xr)\xt]-St^E 



i: 



dSAxt 



= 0, 



for any stopping time r e , and hence, 

St^¥.[Sr\xt\ > E[xl{l-r)\r>t,xt\. 

The last equation confirms that St is indeed the Snell envelope. Consider the following 
solution to the Pree-boundary PDE system: 



g{x,t)=A\{T-tf + 2x^ {T-t) + 



where A is a constant chosen such that g{x, t) — 2x'^{T — t) — {1 — /3){T — tY becomes a 
perfect square. The only possible value for A then is: 



(5 + /?)--^(5 + /?)'-24 



Then an optimal stopping time is the earliest time the reward obtained by stopping now 
equals or exceeds the Snell envelope: 

T* = inf {t:St< 2xl{T - t) + {1 - p){T - tf} , 



= inf < t : x; > 



3{A-1 + (3) 



iT-t)\, 



and the corresponding minimum distortion becomes 

X* (T, (3)^(1 -A) 



^2 



We now examine the problem of choosing optimally a single sample. 

3.3.1. Optimal choice of a single sample. The minimum distortion due to using ex- 
actly one sample is: 



J* (T, 1) = ess inf E 
n 



ess inf E 



/ Xs^ds+ / (xs — Xtj^) ds 

Jo Jti 



Xs^ds + -(T — Ti)'^ds 



= ess inf x(r, 1,Ti) . 

n 

We have thus reduced the optimization problem to one whose solution we already know. 
Hence, we have: 



r* = inf |i : > VS (T - i)| , and, J* (T, 1) ^ (Vs - 
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Figure 5. Parabolic envelopes for the estimation error signal under the 
optimal sampling scheme. There three envelopes starting with the inner- 
most one correspond to the number of remaining samples being 3, 2, and 
1 respectively. 

3.3.2. Optimal multiple sampling. We obtain the family of policies for optimal multi- 
ple sampling by mathematical induction. Suppose that the minimum distortions due 
to using no more than k — 1 samples over [0, T] is given by the sequence of values 

16*1^, . . . , 6'fc_i^|. Then consider the minimal distortion due to using up to k sam- 
ples: 



J* (T, k) = ess inf E 

n 



ess inf E 

n 



Xg ds + 
ess inf x (T, 6'fc_i,ri) . 



x,'^ds + J* {T -Ti, k-l) 
dk-i 



ri 1 
2 ■ 



-(T — Ti) ds 



This proves the hypothesis that the minimum distortions for increasing values of the 

sample budget form a sequence with the form: \6k^\ ■ The last equation also provides 

I ^ J fc>i 

us with the recursion which is started with = 1: 
(9) 





i5 + 9N-l)-^J{5 + 9N-lf -24: 




4 


= ,inf {t : {xt - Xr,f > -fN-k+iT - t} . 




/ 3(6'Ar_i — 6n) 


7. = y 




I -On ■ 





3.4. Comparisons. In figure [6] we have a comparison of the estimation distortions 
incurred by the three sampling strategies on the same time interval [0,T]. The remark- 
able news is that Delta sampling which is optimal for the infinite horizon version of the 
estimation problem is easily beaten by the best deterministic sampling policy. There 
is something structural about Delta sampling which makes it ill suited for finite horizon 
problems with hard budget limits. This also means that it is not safe to settle for 'natural' 
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Performances of the three sampling methods 




Percentage gain of Optimai sampiing of Periodic sampiing 



Number of samples left: N 

(a) Comparisons 




(b) Optimal vs Periodic 



Figure 6. The minimum distortions offered by the three samphng meth- 
ods. As the number of allowed samples grows, the normalized reduction of 
the distortion of optimal sampling from periodic sampling keeps growing 
up to a limit of 67%. 

event-triggered sampling policies such as Delta sampling. Also, notice that the relative 
gain of optimal sampling over periodic sampling consistently grows to about 67%. 

4. Sampling the Ornstein-Uhlenbeck process 

Now we turn to the case when the signal is an Ornstein-Uhlenbeck process: 
(10) dxt = axtdt + dWt, t e [0, T], 

with xo = and Wt being a standard Brownian motion. Again, the sampling times S 
= {ti, . . . , tn} have to be an increasing sequence of stopping times with respect to the 
x-process. They also have to lie within the interval [0,T]. Based on the samples and the 
sample times, the supervisor maintains an estimate waveform xt given by 



(11) 





x^-.e 



a{t-Ti) 



Xj-j^a 



ait-Tjsi) 



if < t < Ti, 

if Ti < t < Tj+i < Tat, 

if < t < T. 



The quality of this estimate is measured by the aggregate squared error distortion: 

r r'^ 

J* (T, N)=E 



'ds 



4.1. Optimal deterministic sampling. Just like in the case of Brownian motion. 
We can show through mathematical induction that uniform sampling on the interval [0, T] 
is the optimal deterministic choice of N samples: For the induction step, we assume that 
the optimal choice of — 1 deterministic samples over [Ti,T2] is the uniform one: 



{di,d2,...dN} = ^Ti + i 



N + 1 

The corresponding minimum distortion becomes: 

1 1 



1,2, 



AT 



(AT + i; 



g^" N+l 



4a2 



2a 



(T2 - TO 
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4.2. Optimal Delta sampling. We do not have an analytical characterization of 
the performance of Delta sampling. Let us first address the single sample case. The 
performance measure then takes the form 



J' 



Thresh 



(T, 1) 



E 



E 



Ci 



a;? + / (xt - xt) dt 



Ci 



Ci 



XtXtdt+ / {xtY dt 



Now notice that the second term can be written as follows 

¥.[xt\J^Ci]^t dt 

where we have used the strong Markov property of Xt-, and that for t > (^i we have 





r r 1 
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/ XtXt dt 


= E 


1/, 















= E 


/ (xtfdt 




Uci J 



the form 



x^e ""^^ ''^^ = Xt- Because of this observation the performance measure takes 





- .T 




Jxhresh (T, 1) = E 


/ xf dt — 


[ {xtfdt 




Jo 


JCi J 



1 - 2aT 



4a2 

-,2aT _ ]^ 



E 
2aT 



,2a(T-Ci) 



X 



4(aT)2 
-2" - 1 + 2a 



C.1 



E 



X 



E 



2a 

2^ g2(aT)(l-Ci/T) _ I 

Y' 2{aT) 

_^ g2a(l-Ci) _ 1 



2a 



where, 

(12) 



t 



t 



aT; Xt 



Xt_ 

T 



T' ' ^ \/T 

We have x satisfying the following SDE: 

dxt = —axtdt + dwt- 

This suggests that, without loss of generality, we can limit ourselves to the normalized 
case T = 1 since the case T ^ 1 can be reduced to the normalized one by using the 
transformations in (12). In fact, we can solve the single sampling problem on [0,1] to 
minimize: 



(13) 



Thresh 



-2a 



1 + 2a 



4a2 



E 



-X 



Ci 



.2a{l-Ci) 

2a~ 



We carry over the definitions for threshold sampling times from section 3.2 We 
do not have series expansions like for the case of the Wiener process. Instead we have a 
computational procedure that involves solving a PDE initial and boundary value problem. 
We have a nested sequence of optimization problems. The choice at each stage being the 
non-zero level 5j. For = 1, the distortion corresponding to a chosen 6i is given by: 



-,2a 



- 1 



1 

2a 



SI 



E [eMi~(i) _ 1] 



,2a 



1 



1 

2a 



SI 



{e^^{l + 2aU\0,0)) -1}, 



4a2 2a 2a Aa"^ 2a 2a 

where the function U^{x,t) defined on [— x [0, 1] satisfies the PDE: 

1 



+ axUl + U} + e 



-2at 



0, 
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along with the boundary and initial conditions: 
'u\-6,,t) = U\6ut)=0 



UHxA] 







for t e [0,1], 
for X e [—5i, 5i] 



We choose the optimal 6i by computing the resultant distortion for increasing values of 6i 
and stopping when the cost stops decreasing and starts increasing. Note that the solution 
U{0, t) to the PDE furnishes us with the performance of the 5i-triggered sampling over 
[t, 1]. We will use this to solve the multiple sampling problem. 

Let the optimal policy of choosing N levels for sampling over [Ti, 1] be given where 
< Ti < 1. Let the resulting distortion be also known as a function of Ti. Let this known 
distortion over [Ti,l] given level-triggered samples be denoted Jxhreshi^ ~ '^i^ 
Then, the + 1 sampling problem can be solved as follows. Let f/^+^(x,t) satisfy the 
PDE: 



along with the boundary and initial conditions: 



t/^+iMi,t) = f/^+i(5i,t) 

U^+HxA)=0 



J, 



Thresh 



t, N) 



for t e [0,1], 
for X e [—5i, 5i] 

Then the distortion we are seeking to minimize over 5i is given by: 



-,2a 



- 1 



4a2 



1 

2a 



5J 
2a 



E 



,2a(l-Ci) 



-,2a 



32a(l-Ci) _ 1 1 _ 



1 

2a 



4a2 

51 



2a 



E[J* 



Thresh 



Ci, N)] 



{e'" (1 + 2at/^(0, 0)) - 1} + f/^+^(0, 0). 



4a2 2a 2a 

We choose the optimal 5i by computing the resultant distortion for increasing values of 
(5i and stopping when the distortion stops decreasing. 

4.3. Optimal Sampling. We do not have analytic expressions for the minimum 
distortion like in the Brownian motion case. We have a numerical computation of the 
minimum distortion by finely discretizing time and solving the discrete-time optimal 
stopping problems. 

By discretizing time, we get random variables xi, . . . ^xm, that satisfy the AR(1) 
model below. For 1 < n < M, 

^2a5 



Xr. 



e^^Xn-i + Wn, m„ ~ AT 



1 



2a 



l<n< M. 



The noise sequence {wn} is i.i.d. and Gaussian. 

Sampling exactly once in discrete time means selecting a sample x,y from the set of 
M + 1 sequentially available random variables xq, . . . ,xm, with the help of a stopping 
time u E {0, 1, ... , M}. We can define the optimum cost to go which can be analyzed as 
follows. For n 



M,M -1 



,0, using equation (13), 



sup E 

n<u<M 



2aS{M-u) _ 1 



X,. 



max < X 



2a 

^2aS{M-n) _ ]^ 

2a 



Xr. 



X 



E[V^^,iXn+l)\x, 



x\ 



The above equation provides a (backward) recurrence relation for the computation of 
the single sampling value function V^{x). Notice that for values of x for which the l.h.s. 
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exceeds the r.h.s. we stop and sample, otherwise we continue to the next time instant. 
We can prove by induction that the optimum pohcy is a time-varying threshold one. 
Specifically for every time n there exists a threshold A„ such that if \xn\ > A„ we sample, 
otherwise we go to the next time instant. The numerical solution of the recursion presents 
no special difficulty if a < 1. If V^{x) is sampled in x then this function is represented 
as a vector. In the same way we can see that the conditional expectation is reduced to a 
simple matrix- vector product. Using this idea we can compute numerically the evolution 
of the threshold Xt with time. The minimum expected distortion for this single sampling 
problem is: 

e^T_i_2aT 
4^^ 

For obtaining the solution to the + 1-sampling problem, we use the solution to the 
iV-sampling problem. For n = M, M — 1, ... 0, 

„2a5(M-u) _ 1 



V^+\x) = sup E 

n<u<M 



2a 



max 



f 2aS{M-n) _ i ^ 

|vf (0) + , d(0) + E [V;Vi(a:„+i)|x. = x] I 



4.4. Comparisons. Figure [7] shows the result of the numerical computations for a 
few stable plants and a single unstable plant. Again, Delta sampling is not very compet- 
itive. In the stable cases, it does provide lower distortion than periodic sampling when 
the size of the sample budget is small. 



5. Summary and extensions 

We have set up the problem of efficient sampling as an optimal multiple sampling prob- 
lem. We characterized the timing channel in adaptive sampling through its modification 
of the estimator. For MMSE estimation of linear systems, under optimal sampling, we 
showed that while the timing channel exists, it affects only the evolution of the statistics 
of the second and higher moments of the estimation error. This reduces the optimization 
problem into a tractable multiple stopping problem. 

We have furnished methods to obtain good sampling policies for the finite horizon state 
estimation problem. When the signal to be kept track of is a Brownian motion, we have 
analytic solutions. When the signal is an Ornstein-Uhlenbeck process, we have provided 
computational recipes to determine the best sampling policies and their performances. 
In both cases. Delta sampling performs poorly with it distortion staying boundedly away 
from zero even as the sample budget increases to infinity. This means that the designer 
cannot just settle for 'natural' event-triggered schemes without further investigation. In 
particular, a scheme optimal in the infinite horizon may be, like Delta sampling, a bad 
performer for the finite horizon. 

The approach adopted in this paper leads us to also consider some sampling and 
filtering problems with multiple sensors. These can possibly be solved in the same way 
as the single sensor problem. The case where the samples are not reliably transmitted 
but can be lost in transmission is computationally more involved. There, the relative 
performances of the three sampling strategies is unknown. However, in principle, the 
best policies and their performances can be computed using nested optimization routines 
like we have used in this paper. 

Another set of unanswered questions involve the performance of these sampling poli- 
cies when the actual objective is not filtering but control or signal detection based on the 
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Estimation distortions for a = -1 



Estimation distortions for a = 




-ir- Best deterministic sampler 
-o- Best level-triggered sampler 
— e — Optimal sampler 



* - Best deterministic sampler 
■□- ' Best level-triggered sampler 
-O — Optimal sampler 




Number of samples allov^ed 



Number of samples allovwed 



(a) A stable case 



(b) A stable case 



Estimation distortions for a = -1 



Estimation distortions for a = 1 



•k - Best deterministic sampler 
■o- ' Best level-triggered sampler 
-©— Optimal sampler 




•k - Best deterministic sampler 
■□- ' Best level-triggered sampler 
-e — Optimal sampler 





Number of samples allov^ed 

(c) A stable case 



Number of samples allowed 



(d) An unstable case 



Figure 7. The minimum distortions offered by the three samphng meth- 
ods. In the stable regime, Delta sampling is more efficient than determin- 
istic sampling when the number of allowed samples is greater than one but 
still small. In the unstable regime, deterministic sampling always beats 
Delta sapling. 

samples. It will be very useful to know the extent to which the overall performance is 
decreased by using sampling designs that achieve merely good filtering performance. The 
communication constraint we treated in this paper was a hard limit on the number of 
allowed samples. Instead, we could use a soft constraint: a limit on the expected number 
of samples. We could also study the effect of mandatory minimum intervals between 
successive sampling times. Extension to nonlinear systems is needed as are extensions to 
the case of partial observations at the sensor. One could follow the attack line sketched 
at the end of section [2l 



A. Optimal sampling with non-standard MMSE estimates 

The statement of lemma [l] and its consequence viz. equation ^ conform to our 
intuition. Was there a real need to investigate the validity of the aforementioned lemma ? 
Here, we give an example of a well-behaved and widely used stochastic process for which, 
the optimal sampling policy leads to an MMSE estimate which is different from that 
under deterministic sampling. For convenience, we consider an infinite horizon repeated 
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sampling problem where the communication constraint is a limit on the average sampling 
rate. 

Choose the state process to be the Poisson counter iVf, a continuous time Markov 
chain. This is a non-decreasing process which starts at zero and takes integer values. Its 
sample paths are piecewise constant and RCLL. The sequence of times between successive 
jumps are independent and are all exponentially distributed with parameter A. 

Under any deterministic sampling rule, the MMSE estimate is piecewise linear with 
slope A: 

(14) Nt = iV,^„,,,, + A (t - di^test) , 

where, diatest is the latest sampling instant as of time t. The optimal sampling policy 
leads to an MMSE estimate which is different. 

Stipulate that the constraint on the average sampling rate is equal to A the parameter 
of the Poisson process. Consider the following sampling policy whose MMSE estimate is 
of the zero-order hold type: 

(15) ^t = A^r,„.., 

= 0, 

(16) { r,+i = inf {t \t > n, Nt > N^, } , > 0, 

= max {Ti\Ti < t} 

This sampling rule with its MMSE estimate Nt leads to an error signal which is identically 
zero. We also have that 

E h+i -Ti] = j,\fi > 0, 

and so, the communication constraint is met. On the other hand, the conventional MMSE 
estimate ( 14 ) would result in a non-zero average squared error distortion. 

Suppose now that the distortion criterion is not the average value of the squared error 
but of the lexicographic distance: 




if Nt = Nt, 
otherwise, 



then, under deterministic sampling, the maximum likelihood estimate: 

minimizes the average lexicographic distortion which will be non-zero. However, the 
adaptive policy (16) provides zero error reconstruction if the constraint on the average 
sampling rate is not less than A. 

B. Statistics of a threshold Hitting time curtailed by a time-out T 

We start by deriving the Moment generating function of the first hitting time r^: 

Ts = inf {t : \xt - Xt\ = 6} , 

with the initial condition xq — Xq = Wo- 

Lemma 3. If ts is the first hitting time of \xt — Xt\ at the threshold 6, then, 
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Proof: Consider the function h{w,t) = e — cosh(\^w) / cosh(\/2s5)] and 
apply Ito calculus on h {wt,t). We can then conclude that 



E[h iwr,,Ts)]-h{wo,0) = E 



[ht{wt, t) + 0.5h^^{wt, t)] dt 



-STS] 



from which we immediately obtain the desired relation because of the boundary condition: 

h{Wrg,Ts) = 0. ■ 

Lemmajs] suggests that the PDF of the random variable ts can be computed as frit) = 
C-^{F^{s)), that is, the inverse Laplace transform of Ft-{s). Invoking the initial condition 
Wo = 0, we can then write 



E[(T 



1 
1 

2^7 



{T-t)U{t)dt- 

T 



{T-t) 



1 



Fr(») 



(T - t)e'' dt 
ds. 



2nj 
ds 



Fr{s)e'' ds 



dt 



cosh(5v2s) 



The previous integral is on a path that includes the whole left half of the complex plane. 

In order to compute this line integral over the complex plane, we need to find the 
poles of the integrand and then apply the residue theorem. Notice first that s = is 
not a pole since the numerator has a double zero at zero. The only poles come from the 
zeros of the function cosh.{5\/2s). Since cosh(x) = cos(jx) we conclude that the zeros of 
cosh((5v^2s) which are also the poles of the integrand are: 



vr 



s, = -(2k + lf — , k 



0,1,2,... 



and they all belong to the negative half plane. This of course implies that they all 
contribute to the integral. We can now apply the residue theorem to conclude that 



E[(T 



1 



1 - sT 



Svrj / s2 cosh(5v2s) 



ds 



fc>0 



1 - SkT 



S - Sk 



lim 

s^-'^k cosh((5v2s) 



In order to find the last limit we can assume that s = Sfc(l + e) and let e — 0. Then, we 
can show that 



S - Sk 

lim 

s^^k cosh(5v2s) 



4si 



(2A; + l)7r' 



Using this expression, the performance measure of the stopping time Tg takes the following 
form: 



J' 



Thresh 



(T, 1) 



' 2k + 1 

k>0 



1 e'"^ - 1 



SkT 



SkT 
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0(A)=1 



TT 



1 + (2A; + 1)2A 



fc>0 



(2fc + i; 



1 - 



TT 



TT 



fc>0 ^ ' 



(-1)' 



TT 



(-1)^ 



k>0 ^ ' k>0 



The final two series in the last equation can be summed explicitly. To do so, we adopt a 
summation technique described in the book of Aigner and Ziegler p[]. Consider: 



dx 



1 + ^' 



"^0 Vfc>0 / 



dx 



E(-i; 



x'^'^dx 



2k + 1 



k>0 ^ k>0 

By an easy evaluation of the definite integral we started with, we get a sum of | for the 

series X]fc>o 2fc+i ' ^^^^ result is useful because the series converges slowly. Proceeding 
along similar lines [7j and working with the multiple integral: 



1 + {XiX2 ■ ■■Xnf 
A 

over the unit hypercube A = [0, 1]" in M", we get an explicit expression for the sum 
J2k>o (2fc+i)" whenever n is an odd number. In particular, 

(-1)' 



k>0 



-1 



TT 



{2k + iy 32' 



E 

A:>0 



(2fc + l)5 1536' 



This reduces the distortion to: 



(17) 



J' 



Thresh 



(T, 1) 



Y 



32A2 



^ ^ ^ (_l)fce-(2fc+i)^ A 
4A ~ A2 ^ (2A; + 1)3 

fc>0 ^ ^ 



where, A = 

The estimation distortion due to using N+1 samples when N is non-negative is given 
through the recursion: 



JThresh (T, N+1) 



rp2 



6^E 



1 



JThresh (T, N) E 



This recursion decomposes the distortion into an expression involving the performance 
of the policies for the last times of the first of + 1 sampling times over the horizon 
[0,T] , and, the infiuence of the first of the sampling times namely, r^^^^. Hence, for our 
recursive solution to the multiple stopping problem, we need to minimize costs like: 



a E 



T{T,a,6)^--6'E[{T-Tsf] 
ositive but no greater than 0.5. 

[(T — T^)^]^ . The evaluation can be done similarly to the one for the first 



2 "/J ^2 

where a is positive but no greater than 0.5. This requires an evaluation of the second 
moment: E 
moment: 



E 



1 

nj 



^ ■ ds. 



s3 cosh {6\/2s) 
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This gives the following expression for the cost T (T, a, 5): 



T(T,a,5) = ^<!0(A) + 



a 



where, A = and we define functions (p,tjj with being the same as it was earlier in 
this part of the appendix: 



= 1 



(2fc + l)3 

A;>0 ^ ^ 



TT 



-D' 



TT 



it>o ^ ^ fe>0 



and, 



"A^ 



TT 



fc>0 



(2A; + i: 



16 (-i)fce-(2fc+i)^A 
7rA2 ^ 



16 



(-1)^ 



16 



-11 



E 



(2/c + ir ^A2^(2A; + l)^ 7rA^(2A; + l)^ ^^2/^ + 1 

After replacing the summable series with their sums, the distortion due to multiple sam- 
ples based on thresholds reduces to the boxed expression below. With A = 

(18) 



T2 



Jxhresh (T, TV + 1) = _ n + ^^^^ 



4 2 
TT TT TT 



.]^\fcg-(2fe+l)2A 



4A A2 ^ (2A; + 1) 

fc>0 ^ ^ 



+ (0.5- JTfe.e./.(T, iV)) 



-Svr^ 7r2 



96A2 2A 



16 V- (-l)^e-(2^'+i)'^ 



7rA2 ^ (2A; + 1)^ 



To characterize the statistics of sample budget utilization by multiple Delta sampling 
we need to find the probabilities of threshold crossings before the time-out T. Given 
the limit of the allowed number of samples N, let Sjv be the random number of samples 
generated under the multiple delta sampling scheme, Then, we have: 

E[Hjv] = • P[t5^ > T] + (1 + E[5jv-i]) -^[ts, < T] , 
= (l + E[S^_i])-(l-P[r,^ <r]). 

As before, we use the Moment generating function of the hitting time to obtain: 

e^^-1 



E[Si] = E 



ds. 



s ■ cosh (5v^) 

With the notation A = and evaluating this complex line integral as in previous cases, 
we obtain: 



(19) 



E[Si] = F[rs^ <T] = 1 



TT 



fe>0 



-1)' 



.--(2A:+1)2A 

2A; + 1 
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